Most of the Ancient City, at least the areas near the major attractions and along the busiest streets, is lined with shops that cater toward tourists. Standard Chinese bric-a-brac is for sale, at relatively high prices. This includes antiques although an alert shopper should beware of the preponderance of forgeries. Bargain hard and keep an eye out for stalls that sell handmade crafts right in the places where they make them. The city makes excellent cloth shoes, which you will see the locals wear as they march up and down the dusty streets. Pingyao also has a large lacquerware manufacturing industry.
If one is looking for nearby modern shops, then outside of west gate is one's best bet.
money
ICBC Industrial and Commercial Bank of China has a branch just outside the West Gate--they have an ATM that accepts foreign cards. When you go through the gate, it's to the right and across the street. checked Oct 2012
There's an ATM of the Agricultural Bank of China a few meters out of the south-western gate of the Ancient City. Foreign Maestro cards accepted checked Aug 2006.
For other options, the nearest modern shops and presumably ATMs will be located outside of West Gate.